Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Maps

With advances in reinforcement learning (RL), agents are now being developed in high-stakes application domains such as healthcare and transportation. Explaining the behavior of these agents is challenging, as the environments in which they act have large state spaces, and their decision-making can be affected by delayed rewards, making it difficult to analyze their behavior. To address this problem, several approaches have been developed. Some approaches attempt to convey the $\textit{global}$ behavior of the agent, describing the actions it takes in different states. Other approaches devised $\textit{local}$ explanations which provide information regarding the agent's decision-making in a particular state. In this paper, we combine global and local explanation methods, and evaluate their joint and separate contributions, providing (to the best of our knowledge) the first user study of combined local and global explanations for RL agents. Specifically, we augment strategy summaries that extract important trajectories of states from simulations of the agent with saliency maps which show what information the agent attends to. Our results show that the choice of what states to include in the summary (global information) strongly affects people's understanding of agents: participants shown summaries that included important states significantly outperformed participants who were presented with agent behavior in a randomly set of chosen world-states. We find mixed results with respect to augmenting demonstrations with saliency maps (local information), as the addition of saliency maps did not significantly improve performance in most cases. However, we do find some evidence that saliency maps can help users better understand what information the agent relies on in its decision making, suggesting avenues for future work that can further improve explanations of RL agents

Information Sharing for Care Coordination

Teamwork and care coordination are of increasing importance to health care delivery and patient safety and health. This paper describes our initial work on developing agents that are able to make intelligent information sharing decisions to support a diverse, evolving team of care providers in constructing and maintaining a shared plan that operates in uncertain environments and over a long time horizon.Engineering and Applied Science

Collaborative Health Care Plan Support

This paper envisions a multi-agent system that assists patients and their health care providers. This system would support a diverse, evolving team in formulating, monitoring and revising a shared "care plan" that operates on multiple time scales in uncertain environments. It would also enhance communication of health information within this planning framework. The coordination of care for children with complex conditions (CCC), which is a compelling societal need, is presented as a model environment in which to develop and assess such systems. The design of algorithms and techniques needed to realize this vision would yield agents capable of being collaborative partners in health care delivery broadly as well as in other environments with similar properties such as rescue and rebuilding after natural disasters. This paper describes the key characteristics of collaborative health care plan support, defines a set of essential capabilities for autonomous "care-augmenting software agents", and discusses three major multi-agents systems research challenges that building such agents raises: evolving long-term plan management, enhancing team interactions, and leveraging human computation for care plan customization.Engineering and Applied Science

An Empirical Evaluation of a Combinatorial Auction for Solving Multi-Agent Pathfinding Problems

Engineering and Applied Science

To Share or not to Share? The Single Agent in a Team Decision Problem

This paper defines the "Single Agent in a Team Decision" (SATD) problem. SATD differs from prior multi-agent communication problems in the assumptions it makes about teammates' knowledge of each other's plans and possible observations. The paper proposes a novel integrated logical-decision-theoretic approach to solving SATD problems, called MDP-PRT. Evaluation of MDP-PRT shows that it outperforms a previously proposed communication mechanism that did not consider the timing of communication and compares favorably with a coordinated Dec-POMDP solution that uses knowledge about all possible observations.Engineering and Applied Science

Deploying AI Methods to Support Collaborative Writing: A Preliminary Investigation

Many documents (e.g., academic papers, government reports) are typically written by multiple authors. While existing tools facilitate and support such collaborative efforts (e.g., Dropbox, Google Docs), these tools lack intelligent information sharing mechanisms. Capabilities such as “track changes” and “diff” visualize changes to authors, but do not distinguish between minor and major edits and do not consider the possible effects of edits on other parts of the document. Drawing collaborators’ attention to specific edits and describing them remains the responsibility of authors. This paper presents our initial work toward the development of a collaborative system that supports multi-author writing. We describe methods for tracking paragraphs, identifying significant edits, and predicting parts of the paper that are likely to require changes as a result of previous edits. Preliminary evaluation of these methods shows promising results.Engineering and Applied Science